1. Field of the Invention
The invention relates to computer-aided music analysis and composition.
2. Description of the Prior Art
Composition and playing of music requires years of dedication to the cause. Many talented individuals are simply unable to dedicate so much of their lives to learning the skill. Technology has grappled with allowing non-practiced individuals to play music for years. Player pianos, automated music and rhythm organs, and electronics keyboards have minimized the learning curve. While these devices automated some parts of music reproduction to some extent, they severely constrained creativity.
The player piano, for example, used a predetermined program indicated by holes in a roll of paper. The keys that were pressed based on those holes were indifferent to the creative ideas of an unskilled operator.
All of these technologies force operators to rely on pre-packaged music originated by others. They allow very little creativity. Even the keynote in which the pre-programmed sounds are to be played is preselected. Merely arranging snippets of another's music has proved a poor substitute for creating one's own music.
Recently, some have tried to apply computer power in aid of the composer. U.S. Pat. No. 5,308,915 is representative of the many systems that use a neural network. Computer-based music analysis and composition has used, for example, neural network computer technology. Neural networks which make use of concepts related to the operation of the human brain. Neural networks operate in an analog or continuously variable fashion. Some neural network approaches use some sort of rule-based preprocessing and post-processing. The knowledge which the system uses to make its decisions is inaccessible to the user.
For example, takes a system with the following steps:
______________________________________ Input from MIDI keyboard (10) .vertline. .vertline. Preprocessor puts input into a form that a neural network can understand (20) .vertline. .vertline. Neural network (30) Postprocessor to turn neural network output back into MIDI (40) .vertline. Output to MIDI sound module (50) ______________________________________
The input and output that the system is sending may be understandable at each point in the process. However, ALL of the LEARNED knowledge that the system uses to make its decisions is hidden in the weights of the connections inside the neural network (30). The inventors recognized that this knowledge is extremely difficult to extract from the network. It is difficult to phrase music in a form directly that can be understood by a network. All neural networks share the common characteristic that at some point in the process, knowledge is not stored in a directly-accessible declarative form.
Another limitation commonly encountered in neural network approaches is related to external feedback, where the output of the network is used at some point in the future as input to the network. Here, the analog nature of the network allows it to slide away from the starting point and toward s one of the melodies on which it was trained. One example is a network which learned the "blue danube". The problem with this network was that no matter what input you gave it, eventually it started playing the blue danube. The key point here is that the network may have learned the blue danube, but it did NOT learn HOW to write it or how to write SIMILAR but not IDENTICAL music.
Moreover, neural networks are analog machines, and it is difficult to make an analog machine (a neural network) approximate a discrete set of data (music with a finite number of pitches and rhythmic positions).
One type of network used for composition is a single feed-forward network. This network has been used to associate chords with melodies. This system was described by Shibata in 1991. This system represents chords as their component tones instead of by their figured bass symbols. The network also required the entire melody at once, meaning it could not be performed in real-time as the melody was being generated by a musician. An important contribution from Shibata's work is the use of psychophysical experiments to gauge the success of a computer compositional approach; listeners evaluated the output of the network compared to a table-driven harmonizing approach and indicated a measure of how natural the output sounded.
Adding recurrent connections to a neural network provides additional computational complexity, and allows the network to evolve some sense of movement through time. This approach has been used to teach a network a single 153-note melody.
The inventors recognized certain limitations in these previous studies. Neural networks have a continuous analog nature, which has proven to be difficult to apply to apply to music's a discrete set of events. Almost all music has some sort of regular rhythm, with notes starting either directly on a beat or at just a simple fraction of the beat. Note durations behave similarly.
Most music is also tonal, using only a finite number of pitch values. Neural networks, which use a continuous or analog mode of operation, require excessive training to approximate this discrete behavior. This is a very inefficient use of a nueral network.
Neural networks learn in a connective way, which is not conducive to determination of the rationale behind the learning. The inventors recognized that a music composer either likes or dislikes certain effects which have been obtained. It is an object of the present invention to allow the composer to interact with the computer based learning system by viewing and/or modifying the results of the computer based learning system. It might be possible to modify a neural network to respond to feedback from a user about what that user likes or dislikes as suggested according to the present invention. Even if this were done, however, it would not be easy to ask the network, "I HATE that| Why did you do that?"
Some research has been done using rule-based computer analyses that learn from examples. Rule-based systems are inherently discrete, easing system training. An example of a generic rule is shown below, with a left-hand side (LHS) referencing one or more attributes A.sub.x and a right-hand side (RHS) referencing an attribute A.sub.RHS. Such a rule inferences the RHS attribute A.sub.RHS. A set of such rules is known as a rule base. ##EQU1##
U.S. Pat. No. 5,418,325 describes a computer receiving a musical element, i.e., a series of notes over time. This is used to build a table of rules that indicate which notes are most likely to follow each note received. Such a table is of some help to a composer of a new element in order to create a series of notes that are pleasing to the ear.
The inventors recognized that this will give a correct distribution, but will not necessarily sound good. Music which is done purely probabalistically is BORING, i.e., it doesn't interest the ear.
U.S. Pat. No. 5,418,323 describes a system in which rules built from a small seed string of notes. The system is usually not responsive to feedback in real-time.
The systems of U.S. Pat. Nos. 5,302,777, 5,218,153, and 4,981,544, for example, create such competing rules but follow through with only simplistic methods of making use of these rules. The present invention defines a new technique of weighing which allows competing rules to be maintained and appropriately used.
It is hence an object of the present invention to provide a system which includes all of the advantageous aspects of the present invention--a system which operates using the least possible amount of computer power to learn musical rules and weights and apply them in real-time. The present invention also allows interaction with the rules, e.g. by viewing and/or modifying the rules that have fired.
The system preferably stores information in the form of rules, unlike the conventional learning system which stores information. The use of rules in addition to learning provides some of the benefits of both. The present invention uses probabilistic rules to obtain many of the capabilities of analog networks. By so doing, the present invention obtains all of the benefits of a rule-based system. This allows us to ask the system to explain its decisions.
Practical operation of these systems is enhanced if the rule base is appropriately managed. Another aspect of the present invention defines a special real-time dependency pruning system which enhances the accuracy of the rulebase. Another aspect teaches segmenting the rulebases in a way which facilitates their use. Yet another aspect of the invention defines using probabilistic, e.g., not deterministic, rules.
The operating techniques used by the present invention allow a simple algorithm with small chunks of data to accompany a live musician. The preferred system uses special rules which are optimized for the use according to the present invention.
It is therefore an object of the invention to provide a music composition system useful to one lacking formal training in musical arts. Another object is to provide a system which creates rules through analysis of music. Another object of the system is to provide a real-time composition system which applies these rules in real-time. The present system does not need to create the rules in real-time. In fact, the computers presently being used take several minutes to create the rules it later is able to apply to musical input with a delay of less than 1/10 second.
Another object of the invention is to provide an automated music composition system that creates rules through real-time analysis of music. In addition, it is an object of the invention to provide an automated music composition system requiring little explicitly-coded knowledge of music. It is a further object of the invention to provide an automated rule-based music composition system in which multiple competing rules contribute to an outcome. Still another object of the invention is to provide an automated rule-based music composition system using special rules optimized to provide the best results.